1. Technical Field
The present disclosure relates to audio and video editing and more specifically to systems and methods for assisting in and automating the mixing and equalizing of multiple audio inputs.
2. Introduction
Audio mixing is the process by which two or more audio signals and/or recordings are combined into a single signal and/or recording. In the process, the source signals' level, frequency content, dynamics, and other parameters are manipulated in order to produce a mix that is more appealing to the listener.
One example of audio mixing is done in a music recording studio as part of the making of an album. During the recording process, the sounds produced by the various instruments and voices are recorded on separate tracks. Oftentimes, the separate tracks have very little amplification or filtering applied to them such that, if left unmodified, the sounds of the instruments may drown out the voice of the singer. Other examples include the loudness of one instrument being greater than another instrument or the sounds from the multiple back-up singers being louder than the single lead singer. Thus, after the recording takes place, the process of mixing the recorded sounds occurs where the various parameters of each source signals are manipulated to create a balanced combination of the sounds that is aesthetically pleasing to the listener.
A similar condition exists during live performances such as at a music concert. In such situations, the sounds produced by each of the singers and musical instruments must be mixed and balanced in real-time before the combined sound signal is transmitted to the speakers and heard by the audience. Tests referred to as “sound checks” often take place prior to the event to ensure the correct balance of each of the sounds. These sorts of tests, however, have difficulty in accounting for the differences in, for example, the ambient sounds that occur before and during a concert. In addition, this type of mixing poses further challenges relating to real-time monitoring and reacting to performance conditions by adjusting of the parameters of each of the audio signals based on the changes in the other signals.
Another example of audio mixing is done during the post-production stage of a film or a television program by which a multitude of recorded sounds are combined into one or more channels. The different recorded sounds may include the dialogue of the actors, the voice-over of a narrator or translator, the ambient sounds, sound effects, and music. Similar to the occurrence in the music recording studio, the mixing step is often necessary to ensure that, for example, the dialogue by the actor or narrator is clearly heard over the ambient noises or background music.
In each of the above-mentioned situations, a mixing console is typically used to conduct the mixing. The mixing console contains multiple inputs for each of the various audio signals and controls for adjusting each signal and one or more outputs having the combined signals. A mixing engineer makes adjustments to each of the input controls while listening to the mixed output until the desired output mix is obtained. More recently, digital audio workstations have been implemented to serve the function of a mixing console.
In addition to the volume control of the entire signal, mixing often applies equalization filters to the signal. Equalization is the process of adjusting the strength of certain frequencies within a signal. For instance, a recording or mixing engineer may use an equalizer to make some high-pitches or frequencies in a vocal part louder while making low-pitches or frequencies in a drum part quieter. The granularity of equalization can range from simple adjustments of treble and boost all the way to having adjustments for every one-third octave. Each of these adjustments, however, require manual inputs and are only as precise as the range of frequencies that it is able to adjust. Once set, the attenuation and gains tend to be fixed for the duration of the recording. In addition, the use of such devices often require the expertise of a trained ear in addition to a good amount of trial and error.
A problem arises when the voice of a singer simultaneously occupies the same frequency range as another instrument. For the purposes of this disclosure, this is known as a “collision.” Due to the physiological limitations of the human ear and the cognitive limits of the human brain, certain combinations of sounds are indistinguishable to a human listener. In addition, some sounds cannot be heard when they follow a louder sound. In such cases, the mix engineer attempts to cancel out certain frequencies of one sound in order for another sound to be heard. The problem with this solution is that an engineer's reaction time and perceptions are based on human cognition and are therefore susceptible to the same errors that are trying to be eliminated.
Thus, there is a perceived need for a solution that performs the mixing in real time or applies a mixing algorithm to one or more audio recording files that would assist in the mixing process.
In addition, it would also be helpful to provide a mixing engineer or other user a visual indication of where the overlaps or collisions occur, to allow for quick identification and corrective adjustments.